Gesture-based user interface method and apparatus

ABSTRACT

A gesture-based user interface method and corresponding apparatus that includes a light source configured to irradiate light to a user, an image sensor configured to receive light reflected from the user and output a depth image of the user, an image processor configured to recognize a user gesture based on the depth image output from the image sensor, and a controller configured to control the light source and the image sensor such that at least one of an optical wavelength of the light source, an optical power level of the light source, a frame rate of the light source and a resolution of the depth image is adjusted according to a gesture recognition mode.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a gesture-based user interface methodand apparatus, and more particularly, to a gesture-based user interfacemethod and apparatus for recognizing a gesture of a user and generatinga command for controlling a content display apparatus.

2. Discussion of the Related Art

A user of a content display apparatus such as a television or a personalcomputer (PC) operates a device such as a remote controller, a keyboardor a mouse so as to interact with a user interface. That is, in order toinput a specific command to a content display apparatus, a user mustoperate a separate device such as a remote controller, a keyboard, amouse or a keypad.

In order to solve inconvenience encountered when a user must operate aseparate input unit so as to control an apparatus and provide a moreintuitive user interface, a gesture-based user interface method andapparatus for recognizing a user gesture using a camera, converting therecognized gesture into a control command, and controlling a contentdisplay apparatus has been proposed.

However, in the gesture-based user interface apparatus of the contentdisplay apparatus, the gesture is recognized under the same conditionswithout considering environments and requirements for recognizing theuser gesture. Accordingly, there is a need for a method of moreaccurately and efficiently recognizing a user gesture according toconditions such as the type of a program or content provided by acontent display apparatus and the position of the user.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a gesture-based userinterface method and apparatus that substantially obviates one or moreproblems due to limitations and disadvantages of the related art.

An object of the present invention is to provide a gesture-based userinterface method and apparatus capable of more accurately andefficiently recognizing a user gesture according to conditions such asthe type of a program or content provided by a content display apparatusand the position of the user.

Another object of the present invention is to provide a gesture-baseduser interface method and apparatus capable of minimizing powerconsumption when recognizing a user gesture.

Another object of the present invention is to provide a gesture-baseduser interface method and apparatus capable of minimizing capacity ofdata transmitted and received when recognizing a user gesture so as tomore rapidly recognize the user gesture.

A further object of the present invention is to provide a gesture-baseduser interface method and apparatus capable of more accuratelyrecognizing a user gesture using two or more camera devices.

Additional advantages, objects, and features of the invention will beset forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of theinvention. The objectives and other advantages of the invention may berealized and attained by the structure particularly pointed out in thewritten description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, agesture-based user interface apparatus includes a light sourceconfigured to irradiate light to a user, an image sensor configured toreceive light reflected from the user and output a depth image of theuser, an image processor configured to recognize a user gesture based onthe depth image output from the image sensor, and a controllerconfigured to control the light source and the image sensor such that atleast one of an optical wavelength of the light source, an optical powerlevel of the light source, a frame rate of the light source and aresolution of the depth image is adjusted according to a gesturerecognition mode.

The gesture recognition mode may include a first mode for recognizing awhole-body gesture and a second mode for recognizing a gesture involvingonly a part of the body, and the recognition range of the second modemay be narrower than that of the first mode.

The controller may control the light source such that the opticalwavelength is set to a first wavelength in the first mode and theoptical wavelength is set to a second wavelength shorter than the firstwavelength in the second mode.

The controller may control the light source such that the frame rate isset to a first frame rate in the first mode and the frame rate is set toa second frame rate greater than the first frame rate in the secondmode.

The controller may control the image sensor such that the resolution ofthe depth image is set to a first resolution in the first mode and theresolution of the depth image is set to a second resolution higher thanthe first resolution in the second mode.

The gesture recognition mode may include a short distance mode in whicha distance between the gesture-based user interface apparatus and theuser is less than a reference distance and a long distance mode in whichthe distance is greater than the reference signal. At this time, thecontroller may set an optical power level to a first power value in theshort distance mode and set the optical power level to a second powervalue greater than the first power value in the long distance mode.

The interface apparatus may further include a gesture-based gesturerecognition information memory configured to store a reference image forrecognizing the user gesture and information regarding a command for auser interface corresponding to the recognized user gesture, and theimage processor may output a command corresponding to the user gesturebased on the reference image and the information regarding the command.

In another aspect of the present invention, a gesture-based userinterface apparatus includes a first depth camera and a second depthcamera, each of which includes a light source for irradiating light to auser and an image sensor for receiving light reflected from the user,converting the light into an electrical image signal and outputting theelectrical image signal, an image processor configured to recognize auser gesture based on the electrical image signal output from the imagesensor, and a controller configured to select at least one camera to beused from among the first depth camera and the second depth cameraaccording to a gesture recognition mode of the user.

The first depth camera may be a zoom camera for accurately photographingthe user and the second depth camera may be a wide angle camera forphotographing a region wider than that of the first depth camera.

The first depth camera and the second depth camera may be set tophotograph gestures of different users.

The first depth camera may be disposed to provide a front image of theuser and the second depth camera may be disposed to provide a side imageof the user.

The first depth camera may be a time of flight (TOF) type camera and thesecond depth camera may be a structured light type camera.

In another aspect of the present invention, a gesture-based userinterface method includes determining a gesture recognition mode of auser, adjusting at least one of an optical wavelength, an optical powerlevel, a frame rate and a resolution of a depth image of the useraccording to the gesture recognition mode, irradiating light to theuser, and receiving light reflected from the user, converting the lightinto the depth image and outputting the depth image.

In another aspect of the present invention, a gesture-based userinterface method includes determining a gesture recognition mode of auser, selecting at least one camera to be used from among a first depthcamera and a second depth camera according to the gesture recognitionmode, irradiating light to the user using the selected at least onecamera, and receiving light reflected from the user, converting thelight into a depth image and outputting the depth image.

In another aspect of the present invention, there is an apparatusconfigured to recognize a user gesture. The apparatus includes: a camerasystem that includes a electromagnetic wave source configured toirradiate electromagnetic waves; and an image sensor configured toreceive a portion of the irradiated electromagnetic waves that isreflected from a user, convert the received electromagnetic waves intoan electrical signal, and output the electrical signal; and a controlleroperatively connected to the camera system, the controller configured toselect at least one of a first field of view and a second field of viewof the camera system according to a gesture recognition mode of theapparatus, the second field of view being wider than the first field ofview.

In another aspect of the present invention, there is method ofrecognizing a user gesture by an apparatus having a light source and animage sensor. The method includes: controlling an operation of theapparatus according to a gesture recognition mode of the apparatus, thegesture recognition mode including settings for controlling at least oneof an optical wavelength of the light source, an optical power level ofthe light source, a frame rate of the light source and an imageresolution of the image sensor The operation of the apparatus includesirradiating light by the light source; receiving a portion of the lightthat is reflected from a user; converting the received light into adepth image of the user by the image sensor; and outputting the depthimage.

In another aspect of the present invention, there is a method ofrecognizing a user gesture by an apparatus including a camera system.The method includes: selecting a field of view of the camera systemaccording to a gesture recognition mode of the apparatus, the selectedfield of view being one of a first field of view and a second field ofview, the second field of view being wider than the first field of view;irradiating electromagnetic waves from the apparatus; receiving, by thecamera system according to the selected field of view, a portion of theirradiated electromagnetic waves that is reflected from a user;converting the received electromagnetic waves into an electrical signal;and outputting the electrical signal.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a schematic diagram showing an example of using agesture-based user interface apparatus according to a first embodimentof the present invention;

FIG. 2 is a block diagram of a gesture-based user interface apparatusand a content display apparatus according to the first embodiment of thepresent invention;

FIG. 3 is a perspective view showing an example of a light source unitof the gesture-based user interface apparatus according to the firstembodiment of the present invention;

FIGS. 4 and 5 are schematic diagrams showing a user operation in a firstmode;

FIGS. 6 and 7 are schematic diagrams showing a user operation in asecond mode;

FIGS. 8 to 10 are diagrams showing depth images having differentresolutions;

FIG. 11 is a schematic diagram showing a positional relationship betweena user and a camera in a short distance mode;

FIG. 12 is a schematic diagram showing a positional relationship betweena user and a camera in a long distance mode;

FIG. 13 is a flowchart illustrating a gesture-based user interfacemethod according to the first embodiment of the present invention;

FIG. 14 is a schematic diagram showing a first example of using agesture-based user interface apparatus according to a second embodimentof the present invention;

FIG. 15 is a block diagram of the gesture-based user interface apparatusand a content display apparatus according to the second embodiment ofthe present invention;

FIG. 16 is a schematic diagram showing a second example of using thegesture-based user interface apparatus according to the secondembodiment of the present invention;

FIG. 17 is a schematic diagram showing a third example of using thegesture-based user interface apparatus according to the secondembodiment of the present invention; and

FIG. 18 is a flowchart illustrating a gesture-based user interfacemethod according to the second embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. FIG. 1 is a schematic diagram showing an example of using agesture-based user interface apparatus (hereinafter, referred to as aninterface apparatus) according to a first embodiment of the presentinvention.

As shown in FIG. 1, the interface apparatus 10 according to the presentinvention is connected to a content display apparatus 20 and ispositioned so as to capture a user gesture. The interface apparatus 10recognizes the gesture from the image of the user, converts the gestureinto a command for controlling the content display apparatus 20, andenables the content display apparatus to perform a function desired bythe user 30.

FIG. 2 is a block diagram of the gesture-based user interface apparatus10 and the content display apparatus 20 according to the firstembodiment of the present invention. Hereinafter, the configurationexample of the interface apparatus 10 and the content display apparatus20 will be described in detail with reference to FIG. 2.

The interface apparatus 10 includes a first light source 100, an imagesensor 104, a camera controller 106, an image processor 108, and agesture recognition information memory 110. Optionally, the interfaceapparatus 10 may include a second light source 102 havingcharacteristics different from those of the first light source 100.

The first and second light sources 100 and 102 irradiate light to theuser in order to acquire the image of the user. The first and secondlight sources 100 and 102 may include infrared light (IR) light sources.The first and second light sources 100 and 102 may have differentwavelengths. For example, the first light source 100 may be an IR lightsource having a wavelength of 780 nm and the second light source 102 maybe an IR light source having a wavelength of 850 nm, which is longerthan that of the first light source 100.

FIG. 3 is a perspective view showing an example of a light source unitof the gesture-based user interface apparatus 10 according to the firstembodiment of the present invention. As shown in FIG. 3, the lightsource unit may include first light sources 100 disposed on a centralportion thereof and second light sources 102 arranged to surround thefirst light sources 100. The first light sources 100 and the secondlight sources 102 may have different characteristics in terms ofwavelength, an optical power level, frame rate, etc.

The image sensor 104 receives the light reflected from the user andgenerates and outputs a depth image of the user. In the presentinvention, the depth image may be generated using a time of flight (TOF)method or a structured light method.

In the TOF method, distance information between the user and the imagesensor 104 is acquired from a phase difference between light emittedfrom the light source and the light reflected from the user.

In the structured light method, infrared ray patterns (numerous infraredray points) are emitted from the light source, the emitted patterns arereflected from the user, the reflected patterns are captured by theimage sensor 104 including a filter, and the distance informationbetween the user and the image sensor is acquired based on distortion ofthe patterns. In the structured light method, a separate processor isnecessary in addition to the image sensor 104, in order to acquire adepth image. Accordingly, the interface apparatus 10 according to thepresent invention may further include an additional processor forgenerating a depth image from the light received by the image sensor104.

The image sensor 104 may include, for example, a complementary metaloxide semiconductor (CMOS) sensor or a charge coupled device (CCD)sensor.

The image processor 108 recognizes the user gesture based on the depthimage output from the image sensor 104. For example, the image processor108 detects at least one object having a peak value from a first depthimage acquired by the image sensor 104. In order to detect the peakvalue, various methods may be used.

More specifically, the image processor 108 may compute a mean m and anaverage absolute deviation σ of pixel values in the depth image. Amedian and an average absolute deviation from the median may be used asthe mean and the average absolute deviation.

The image processor 108 may generate a binary image having the samesize, in which a value of 1 is allocated to all pixels having a depthvalue higher than a threshold (m+Kσ) and a value of 0 is allocated tothe other pixels. K may vary according to noise levels and the number ofobjects displayed in the image. In the binary image, connectedcomponents may be identified as one object and a unique ID may beapplied thereto.

Similarly, the image processor 108 detects at least one object having apeak value from a second depth image. The image processor 108 comparesthe coordinate of at least one object extracted from the second depthimage with the coordinate of at least one object extracted from thefirst depth image and sets at least one object having a changedcoordinate as a target object.

By comparing the two depth images, a mobile object is set as a targetobject and a stationary object is not set as a target object.Accordingly, the image processor 108 may distinguish between a body partsuch as a hand of the mobile user and a stationary object and set thehand of the mobile user as a target object.

At least one object set as the target object may be used as an inputunit of the user interface. For example, the user may input a gestureusing the target object and the image processor 108 analyzes the inputgesture and outputs a command according to the result of the analysis.

That is, if the user moves a body part set as a target object, themovement of the body part may be photographed using the light source andthe image sensor 104, the photographed depth image may be analyzed, anda pointer displayed on the content display apparatus 20 may be moved. Inaddition to the movement of the pointer, various functions of thecontent display apparatus, such as channel up/down or volume up/down maybe performed.

At this time, a command table in which gestures of a specific user aremapped to commands corresponding thereto may be stored in the gesturerecognition information memory 110. The image processor 108 may output acommand corresponding to the user gesture to the content displayapparatus 20 based on the command table stored in the gesturerecognition information memory 110. At this time, a mapping relationshipbetween the command defined in the command table and the gesture may bearbitrarily set by the user.

Although the image processor 108 and the gesture recognition informationmemory 110 are shown as components belonging to the interface apparatus10 of the present embodiment in FIG. 2, the image processor 108 and thegesture recognition information memory 110 may be included in thecontent display apparatus 20 such as the television or the PC so as toperform the same function.

The camera controller 106 controls the light source and the image sensor104 so as to control at least one of an optical wavelength of the lightsource, an optical power level of the light source, a frame rate of thelight source and resolution of the depth image according to a gesturerecognition mode.

The gesture recognition mode may be divided into a first mode forrecognizing a whole-body gesture and a second mode for recognizing agesture involving only a part of the body. The recognition range of thesecond mode is narrower than that of the first mode.

For example, as shown in FIG. 4, if content which is being reproduced bythe content display apparatus 20, such as TV broadcasts, movies ore-books, does not require a complicated operation in terms of control,the gesture recognition mode may be set to the first mode. Since thefirst mode requires relatively simple control such as page turning,channel change or volume change, as shown in FIG. 5, the user maycontrol the content display apparatus 20 using a relatively largegesture in the first mode.

As shown in FIG. 6, if content which is being reproduced by the contentdisplay apparatus 20, such as html documents used in a PC or games,requires an accurate operation in terms of control, the gesturerecognition mode may be set to the second mode. Since the second moderequires complicated commands such as selection, clicking and copying ofa displayed object such as a text, as shown in FIG. 6, the user cancontrol the content display apparatus 20 using a relatively smallgesture such as movement of a finger in the second mode.

The user makes a control intention using a specific gesture to thecontent display apparatus 20. The user may acquire control rights to thegesture-based user interface. At this time, the first mode is used untilthe user acquires control rights, the user gains the right to controlthe content display apparatus 20 based upon the control rights, and thesecond mode is set once the control rights expire.

The camera controller 106 may receive information regarding the type ofa program or content which is being reproduced from a controller 206 ofthe content display apparatus 20 and set the gesture recognition mode tothe first mode or the second mode. The camera controller 106 maydistinguish 3 or more gesture recognition modes according tocharacteristics of the content.

In the set gesture recognition modes, the camera controller 106 maycontrol the light source so as to set an optical wavelength to a firstwavelength in the first mode and to set an optical wavelength to asecond wavelength shorter than the first wavelength in the second moderequiring accurate gesture recognition.

For example, the camera controller 106 may control the second lightsource 102 having a wavelength of 850 nm to be used to acquire a depthimage in the first mode which does not require accurate gesturerecognition. In this case, by using the second light source 102 havingthe wavelength longer than that of the first light source 101, it ispossible to reduce energy consumption. In contrast, the cameracontroller 106 may control the first light source 100 having awavelength of 780 nm to be used to acquire a more accurate depth imagein the second mode requiring accurate gesture recognition.

The camera controller 106 may control the light source such that theframe rate is set to a first frame rate in the first mode and the framerate is set to a second frame rate greater than the first frame rate inthe second mode. For example, the camera controller 106 may control thelight source to irradiate light at a frame rate of 15 fps so as toacquire a depth image in the first mode which does not require accurategesture recognition. In contrast, the camera controller 106 may controlthe light source to irradiate light at a frame rate of 60 fps so as toacquire a more accurate depth image in the second mode which requiresaccurate gesture recognition.

The camera controller 106 may control the image sensor 104 such that theresolution of the depth image is set to a first resolution in the firstmode and is set to a second resolution higher than the first resolutionin the second mode. In the first mode, the gesture can be more rapidlyrecognized by reducing the output data amount of the depth image.

FIGS. 8 to 10 show examples of depth images having differentresolutions. FIG. 8 shows a depth image having a VGA resolution(640×480), FIG. 9 shows a depth image having a QVGA resolution(320×240), and FIG. 10 shows a depth image having a QQVGA resolution(160×120).

For example, the camera controller 106 may control the image sensor 104to output a depth image having a low resolution of QQVGA (160×120) tothe image processor 108 as shown in FIG. 10 in the first mode which doesnot require accurate gesture recognition. In contrast, the cameracontroller 106 may control the image sensor 104 to output a depth imagehaving a high resolution of VGA (640×480) as shown in FIG. 8 in thesecond mode requiring accurate gesture recognition. If the gesturerecognition mode is divided into three modes according to programcharacteristics, it is possible to respectively set resolutionsaccording to the modes.

The camera controller 106 may divide the gesture recognition mode into ashort distance mode indicating that a distance between the interfaceapparatus 10 and the user is less than a reference distance and a longdistance mode indicating that the distance between the interfaceapparatus 10 and the user is greater than the reference distance.

FIG. 11 is a schematic diagram showing a positional relationship betweena user and a camera in a short distance mode, and FIG. 12 is a schematicdiagram showing a positional relationship between a user and a camera ina long distance mode.

If the user 30 is separated from the interface apparatus 10 by arelatively short distance d₁ as shown in FIG. 11, the camera controller106 sets the gesture recognition mode to the short distance mode and, ifthe user 30 is separated from the interface apparatus 10 by a relativelylong distance d₂ as shown in FIG. 12, the camera controller 106 sets thegesture recognition mode to the long distance mode.

The camera controller 106 may set an optical power level to a firstpower value in the short distance mode and set the optical power levelto a second power value greater than the first power value in the longdistance mode. In the short distance mode, the light source is set tohave a relatively low power level so as to reduce power consumption. Incontrast, in the long distance mode, the light source is set to have arelatively high power level so as to increase the recognition rate ofthe user gesture.

For example, the camera controller 106 sets the power level of the lightsource to 200 Lux in the short distance mode and sets the power level ofthe light source to 800 Lux in the long distance mode, therebyincreasing a recognizable distance.

The content display apparatus 20 to which the above-described interfaceapparatus 10 is applied includes all apparatuses for reproducingmultimedia content, such as a digital television or a PC. In the presentembodiment, a digital television is shown as an example of the contentdisplay apparatus 20.

Accordingly, the content display apparatus 20 includes a tuner 200, ademodulator 202, an external device interface 210, a network interface204, a memory 214, a user interface 208, a controller 206, a display212, an audio output unit 215 and a power supply 216.

The tuner 200 a) tunes to a Radio Frequency (RF) broadcast signalcorresponding to a channel selected by a user or a channel stored inadvance from among RF broadcast signals received through an antenna, andb) converts the tuned RF broadcast signal into an Intermediate Frequency(IF) signal or a baseband video or audio signal.

The tuner 200 may receive an RF broadcast signal of a single carrieraccording to an Advanced Television System Committee (ATSC) scheme andan RF broadcast signal of multiple carriers according to a Digital VideoBroadcasting (DVB) scheme.

The tuner 200 may sequentially tune to RF broadcast signals of allbroadcast channels stored through a channel storage function from amongthe received RF broadcast signals and convert the tuned RF broadcastsignals into IF signals or baseband video or audio signals.

The demodulator 202 receives a digital IF (DIF) signal converted by thetuner 200 and demodulates the DIF signal.

For example, if the DIF signal output from the tuner 200 is based on anATSC scheme, the demodulator 202 performs 8-vestigial side banddemodulation 8-VSB. The demodulator 202 may perform channel decoding. Inorder to perform channel decoding, the demodulator 202 includes atrellis decoder, a de-interleaver and a Reed-Solomon decoder so as toperform trellis decoding, de-interleaving and Reed-Solomon decoding.

The demodulator 202 may perform demodulation and channel decoding andoutput a transport stream (TS). At this time, the TS may be a signal inwhich a video signal, an audio signal and a data signal are multiplexed.By way of example, the TS will be assumed to be an MPEG-2 TS in which anMPEG-2 video signal and a Dolby AC-3 audio signal are multiplexed. Morespecifically, the MPEG-2 TS may include a 4-byte header and a 184-bytepayload.

The demodulator 202 may include an ATSC demodulator and a DVBdemodulator.

The TS output from the demodulator 202 is input to the controller 206.The controller 206 performs demultiplexing, video/audio signalprocessing, etc., outputs video to the display 212, and outputs audio toan audio output unit 215.

The external device interface 210 is configured to transmit or receivedata to or from an external device. The external device interface 210may include an A/V input/output unit (not shown) and a wirelesscommunication unit (not shown).

For example, the external device interface 210 may be connected to anexternal device such as a Digital Versatile Disc (DVD), Blu-ray, gameconsole, camcorders, or (laptop) computers in a wired/wireless manner.The external device interface 210 sends a video, audio or data signalreceived from the connected external device to the controller 206 of thecontent display apparatus 20. In addition, the external device interface210 may output the video, audio or data signal processed by thecontroller 206 to the connected external device. The external deviceinterface 210 may include an A/V input/output unit (not shown) or awireless communication unit (not shown).

At this time, the A/V input/output unit inputs the video signal and theaudio signal of the external device to the display apparatus 20 and mayinclude a Universal Serial Bus (USB) port, a Composite Video BankingSync (CVBS) terminal, a component terminal, an S-video terminal(analog), a Digital Visual Interface (DVI) terminal, a High DefinitionMultimedia Interface (HDMI) terminal, an RGB terminal, and a D-SUBterminal.

In addition, the external device interface 210 may be connected tovarious set-top boxes through at least one of the above-describedvarious terminals so as to perform an input/output operation with theset-top boxes. The external device interface 210 may transmit or receivedata to or from a supplementary 3D display.

The network interface 204 provides an interface for connecting thecontent display apparatus 20 to a wired/wireless network including anInternet network. The network interface 204 may include an Ethernetport, for connection with a wired network. For connection with awireless network, a communication standard such as Wireless LAN (WLAN)(Wi-Fi), Wireless Broadband (WiBro), World Interoperability forMicrowave Access (Wimax), or High Speed Downlink Packet Access (HSDPA)may be used.

The network interface 204 is configured to receive content or dataprovided by an Internet or content provider or a network manager over anetwork. That is, the network interface 204 may receive content such asmovies, advertisements, games, VOD, or broadcast signals and informationassociated therewith provided by the content provider. In addition, thenetwork interface 204 may receive update information and firmwareupdates provided by the network manager. In addition, the networkinterface 204 may transmit data to the Internet or content provider orthe network manager.

In addition, the network interface 204 is connected to, for example, anInternet Protocol TV (IPTV) so as to receive and transmit a video, audioor data signal processed by a set-top box for IPTV to the controller206, and to transmit signals processed by the controller 206 to theset-top box for IPTV, in order to perform bidirectional communication.

The IPTV may include an ADSL-TV, VDSL-TV, FFTH-TV or the like accordingto the type of transmission network or include TV over DSL, Video overDSL, TV over IP (TVoIP), Broadband TV (BTV), or the like. In addition,the IPTV may include an Internet TV capable of Internet access or afull-browsing TV.

The memory 214 may store a program in the controller 206 for performingsignal processing and control and store a processed video, audio or datasignal.

In addition, the memory 214 may perform a function for temporarilystoring a video, audio or data signal input through the external deviceinterface 210. In addition, the memory 214 may store information aboutpredetermined broadcast channels through a channel storage function suchas a channel map.

The memory 214 may include at least one of a flash memory type storagemedium, a hard disk type storage medium, a multimedia card micro typemedium, a card type memory (e.g., SD memory, XD memory, or the like), aRAM, or a ROM (EEPROM or the like). The content display apparatus 20 mayreproduce and provide a file (a moving image file, a still image file, amusic file, a document file, or the like) stored in the memory 214 tothe user.

Although FIG. 2 shows an embodiment in which the memory 214 is includedseparately from the controller 206, the present invention is not limitedthereto and the memory 214 may be included in the controller 206.

The user interface 208 sends a signal input by the user to thecontroller 206 through a gesture or sends a signal from the controller206 to the user.

For example, the user interface 208 may receive a user input signal,such as power on/off, channel selection or screen setup, from theinterface apparatus 10 or transmit a control signal from the controller206 to the interface apparatus 10 or 210.

In addition, for example, the user interface 208 may send a user inputsignal input through a local key (not shown) such as a power key, achannel key, a volume key, or a setup value to the controller 206.

In addition, for example, the user interface 208 may receive a signalinput by the user using various methods and send the signal to thecontroller 206 or receive a signal from the controller 206. The userinterface 208 may receive the signal input by the user using a remotecontroller, a touch sensor, a voice sensor or the like.

The controller 206 may demultiplex the TS input through the tuner 200,the demodulator 202 or the external device interface 210 or process thedemultiplexed signals, and generate and output signals for a video oraudio output.

The video signal processed by the controller 206 may be input to thedisplay 212 such that a video corresponding to the video signal isdisplayed. The video signal processed by the controller 206 may be inputto an external output device through the external device interface 210.

The audio signal processed by the controller 206 may be audibly outputthrough the audio output unit 215. In addition, the audio signalprocessed by the controller 206 may be input to an external outputdevice through the external device interface 210.

The controller 206 may control the overall operation of the contentdisplay apparatus 20. For example, the controller 206 may control thetuner 200 so as to tune to an RF broadcast corresponding to a channelselected by the user or a channel stored in advance.

In addition, the controller 206 may control the content displayapparatus 20 according to a user command input through the userinterface 208 or an internal program.

For example, the controller 206 controls the tuner 200 such that thesignal of a channel selected according to a predetermined channelselection command received through the user interface 208 is input. Thevideo, audio or data signal of the selected channel is processed. Thecontroller 206 may output information about the channel selected by theuser through the display 212 or the audio output unit 215 together withthe video or audio signal.

As another example, the controller 206 may control a video or audiosignal received from an external device, for example, a camera or acamcorder, through the external device interface 210 to be outputthrough the display 212 or the audio output unit 215 according to anexternal device video reproduction command received through the userinterface 208.

The controller 206 may control the display 212 to display an image. Forexample, the controller may control a broadcast image input through thetuner 200, an external input image input through the external deviceinterface 210, an image input through a network interface 204, or animage stored in the memory 214 to be displayed on the display 212.

At this time, the image displayed on the display 212 may be a stillimage, a moving image, a 2D image or a 3D image.

The controller 206 generates and displays a predetermined object in theimage displayed on the display 212 as a 3D object. For example, theobject may be at least one of a connected web screen (newspaper,magazine, or the like), an Electronic Program Guide (EPG), variousmenus, a widget, an icon, a still image, a moving image, or text.

The controller 206 provides a table indicating a relationship betweengestures and commands corresponding thereto to the gesture recognitioninformation memory 110 of the interface apparatus 10. The controller 206provides information regarding the type of a program or content to thecamera controller 106. In addition, the controller 206 may directlydetermine the gesture recognition mode based on the informationregarding the type of the program or content and provide the gesturerecognition mode to the controller 106.

The display 212 converts a video signal, a data signal, an OSD signal ora control signal processed by the controller 206 or a video signal, datasignal or a control signal received through the external deviceinterface 210 and generates a drive signal.

The display 212 may include a Plasma Display Panel (PDP), a LiquidCrystal Display (LCD), an Organic Light Emitting Diode (OLED) display,and a flexible display, which are capable of displaying a 3D image.

In the embodiment of the present invention, for 3D image viewing, thedisplay 212 may be divided into a supplementary display type and asingle display type.

The display 212 may include a touch screen and function as an inputdevice as well as an output device.

The audio output unit 215 receives the audio signal processed by thecontroller 206, for example, a stereo signal, a 3.1 channel signal or a5.1 channel signal, and outputs audio. The audio output unit 215 may beimplemented by various types of speakers.

The content display apparatus 20 may be a fixed digital broadcastreceiver capable of receiving at least one of an ATSC (8-VSB) digitalbroadcast, a DVB-T (COFDM) digital broadcast, or an ISDB-T (BST-OFDM)digital broadcast or a mobile digital broadcast receiver capable ofreceiving at least one of a terrestrial DMB digital broadcast, asatellite DMB digital broadcast, an ATSC-M/H digital broadcast, a DVB-H(COFDM) digital broadcast or a Media Forward Link Only digitalbroadcast. In addition, the content display apparatus 20 may be a cable,satellite or IPTV digital broadcast receiver.

The content display apparatus 20 shown in FIG. 2 is only exemplary andthe components may be combined, added or omitted according to the typeof the content display apparatus. That is, two or more components may becombined into one component or one component may be divided into two ormore components, as necessary. The functions of the blocks are purelyexemplary and do not limit the scope of the present invention.

Hereinafter, the gesture-based user interface method according to thefirst embodiment of the present invention using the above-described userinterface apparatus 10 will be described in detail. FIG. 13 is aflowchart illustrating the gesture-based user interface method accordingto the first embodiment of the present invention.

First, a gesture recognition mode of a user is determined (S100). Thegesture recognition mode may be divided into a first mode forrecognizing a whole-body gesture and a second mode for recognizing agesture involving only a part of the body. The recognition range of thesecond mode is narrower than that of the first mode.

For example, as shown in FIG. 4, if content which is being reproduced bythe content display apparatus 20, such as TV broadcasts, movies ore-books, does not require a complicated operation in terms of control,the gesture recognition mode may be set to the first mode. Since thefirst mode requires relatively simple control such as page turning,channel change or volume change, as shown in FIG. 5, the user maycontrol the content display apparatus 20 using a relatively largegesture in the first mode.

As shown in FIG. 6, if content which is being reproduced by the contentdisplay apparatus 20, such as html documents used in a PC or games,requires accurate operation in terms of control, the gesture recognitionmode may be set to the second mode. Since the second mode requiresaccurate complicated commands such as selection, clicking and copying ofa displayed object such as text, as shown in FIG. 6, the user cancontrol the content display apparatus 20 using a relatively smallaccurate gesture such as movement of a finger in the second mode. Atthis time, the gesture recognition mode may be divided into 3 or moregesture recognition modes according to content characteristics.

If the user is separated from the interface apparatus 10 by a relativelysmall distance d₁ as shown in FIG. 11, the camera controller 106 setsthe gesture recognition mode to the short distance mode and, if the useris separated from the interface apparatus 10 by a relatively largedistance d₂ as shown in FIG. 12, the camera controller 106 sets thegesture recognition mode to the long distance mode.

Next, conditions of the light source are set according to the gesturerecognition mode (S110). That is, at least one of optical wavelength,optical power level and frame rate is controlled according to the setgesture recognition mode.

In the set gesture recognition mode, the optical wavelength is set to afirst wavelength in the first mode and the optical wavelength is set toa second wavelength shorter than the first wavelength in the secondmode. For example, in the first mode which does not require accurategesture recognition, a depth image may be acquired using the secondlight source 102 having a wavelength of 850 nm. In this case, by usingthe second light source 102 having the wavelength longer than that ofthe first light source 101, it is possible to reduce energy consumption.In contrast, in the second mode requiring accurate gesture recognition,a more accurate depth image may be acquired using the first light source100 having a wavelength of 780 nm.

The light source may be controlled such that the frame rate is set to afirst frame rate in the first mode and the frame rate is set to a secondframe rate that is greater than the first frame rate. For example, inthe first mode which does not require accurate gesture recognition,light is irradiated at a frame rate of 15 fps so as to acquire a depthimage. In contrast, in the second mode which requires accurate gesturerecognition, light is irradiated at a frame rate of 60 fps so as toacquire a more accurate depth image.

If the gesture recognition mode is divided into a short distance modeand a long distance mode as described above, the optical power level isset to a first power value in the short distance mode and the opticalpower level is set to a second power value greater than the first powervalue in the long distance mode. In the short distance mode, the lightsource is set to have a relatively low power level so as to reduce powerconsumption. In contrast, in the long distance mode, the light source isset to have a relatively high power level so as to increase the gesturerecognition rate.

For example, the camera controller 106 sets the power level of the lightsource to 200 Lux in the short distance mode and sets the power level ofthe light source to 800 Lux in the long distance mode so as to increaserecognition range.

Next, the resolution of the depth image is set according to the gesturerecognition mode (S120). That is, the resolution of the depth image isset to a first resolution in the first mode and the resolution of thedepth image is set to a second resolution higher than the firstresolution in the second mode. Thus, in the first mode, the gesture canbe more rapidly recognized by reducing the output data amount of thedepth image.

FIGS. 8 to 10 show examples of depth images having differentresolutions. FIG. 8 shows a depth image having a VGA resolution(640×480), FIG. 9 shows a depth image having a QVGA resolution(320×240), and FIG. 10 shows a depth image having a QQVGA resolution(160×120).

For example, the camera controller 106 may control the image sensor 104to output a depth image having a low resolution of QQVGA (160×120) tothe image processor 108 in the first mode which does not requireaccurate gesture recognition. In contrast, the camera controller 106 maycontrol the image sensor 104 to output a depth image having a highresolution of VGA (640×480) in the second mode requiring accurategesture recognition. If the gesture recognition mode is divided intothree modes according to program characteristics, it is possible torespectively set resolutions according to the modes.

Next, light is irradiated to the user, the light reflected from the useris received by the image sensor 104, and the depth image is acquired(S130). The image sensor 104 receives the light reflected from the userand generates and outputs the depth image of the user. As describedabove, the depth image may be generated using a time of flight (TOF)method or a structured light method.

Next, the user gesture is recognized based on the depth image using theimage processor (S140). For example, at least one object having a peakvalue is detected from a first depth image acquired by the image sensor104. In order to detect the peak value, various methods may be used.

More specifically, a mean m and an average absolute deviation σ of pixelvalues may be computed in the depth image. A median and an averageabsolute deviation from the median may be used as the mean and theaverage absolute deviation.

It is possible to generate a binary image having the same size, in whicha value of 1 is allocated to all pixels having a depth value higher thana threshold (m+Kσ) and a value of 0 is allocated to the other pixels. Kmay vary according to noise levels and the number of objects in theimage. In the binary image, connected components may be identified asone object and a unique ID may be applied thereto.

Similarly, at least one object having a peak value is detected from asecond depth image. The coordinate of at least one object extracted fromthe second depth image is compared with the coordinate of at least oneobject extracted from the first depth image and at least one objecthaving a changed coordinate is set as a target object.

By comparison between the two depth images, a mobile object is set as atarget object and a stationary object is not set as a target object.Accordingly, a mobile user's hand and a stationary object aredistinguished and the mobile user's hand is set as a target object.

At least one object set as the target object may be used as an inputunit of the user interface. For example, the user may input a gestureusing the target object. The input gesture is analyzed.

Next, a control command corresponding to the recognized user gesture isoutput (S150). That is, if the user moves a body part set as a targetobject, the movement of the body part may be photographed using thelight source and the image sensor 104, the photographed depth image maybe analyzed, and a pointer displayed on the content display apparatus 20may be moved. In addition to the movement of the pointer, variousfunctions of the content display apparatus, such as channel up/down orvolume up/down may be performed.

At this time, a command table in which gestures of a specific user aremapped to commands corresponding thereto may be stored. A commandcorresponding to the user gesture may be output based on the commandtable. At this time, a mapping relationship between the command definedin the command table and the gesture may be arbitrarily set by the user.

Next, the function corresponding to the control command is performedwith respect to the content display apparatus 20 (S160).

Hereinafter, a gesture-based user interface apparatus 12 and methodaccording to a second embodiment of the present invention will bedescribed. FIG. 14 is a schematic diagram showing an example of usingthe gesture-based user interface apparatus 12 according to the secondembodiment of the present invention.

As shown in FIG. 14, the interface apparatus 12 according to the presentembodiment is connected to a content display apparatus 60 and ispositioned so as to capture a user gesture. At this time, the interfaceapparatus 12 recognizes a gesture from a user image using two depthcameras 40 and 50, converts the gesture into a command for controllingthe content display apparatus 60, and enables the content displayapparatus to perform a function desired by the user.

A first depth camera 50 may be a zoom camera for accuratelyphotographing the user gesture and a second depth camera 40 may be awide angle camera for photographing a region wider than that of thefirst depth camera 50.

The first depth camera 50 may be a TOF type camera and the second depthcamera 40 may be a structured light type camera according to the methodof acquiring a depth image.

In this case, one or both of the first depth camera 50 and the seconddepth camera 40 may be used according to the above-described gesturerecognition mode.

FIG. 15 is a block diagram of the gesture-based user interface apparatus12 and the content display apparatus 60 according to the secondembodiment of the present invention. As shown in FIG. 15, the interfaceapparatus 12 according to the present embodiment includes the firstdepth camera 50 and the second depth camera 40.

The first depth camera 50 and the second depth camera 40 include lightsources 400 and 500 for irradiating light to the user and image sensors402 and 502 for receiving the light reflected from the user, convertingthe light into depth images and outputting the depth images,respectively.

If the first depth camera 50 and the second depth camera 40 aresimultaneously used, the lights output from the light sources 400 and500 interfere with each other. Accordingly, in order to prevent suchinterference, optical power output timings of the first depth camera 50and the second depth camera 40 may be adjusted through communicationtherebetween.

In FIG. 15, unlike FIG. 2, an image processor 618 and a gesturerecognition information memory 620 are included in the content displayapparatus 60 so as to perform the function. However, alternatively, theimage processor 618 and the gesture recognition information memory 620may be included in the first depth camera 50 and the second depth camera40, respectively.

As the functions and the configurations of the light sources 400 and500, the image sensors 402 and 502, the camera controllers 406 and 506,the image processor 618, the gesture recognition information memory 620and the components included in the content display apparatus 60 areequal to those of the first embodiment, a detailed description thereofwill be omitted.

The controller 606 of the content display apparatus 60 selects at leastone of the first depth camera 50 and the second depth camera 40according to the gesture recognition mode.

As described in the first embodiment, the gesture recognition mode maybe divided into a first mode for recognizing a whole-body gesture and asecond mode for recognizing a gesture involving only a part of the body.The recognition range of the second mode is narrower than that of thefirst mode.

The controller 606 of the content display apparatus 60 may set thegesture recognition mode to the first mode or the second mode accordingto information regarding the type of a program or content which is beingreproduced. In addition, the controller 606 may set the gesturerecognition mode to a third mode according to content characteristics.The controller 606 may set a camera to be used according to the gesturerecognition mode.

For example, the controller 606 may control only the second depth camera40 which is the wide angle camera to be used in the first mode andcontrol only the first depth camera 50 which is the zoom camera to beused in the second mode. In a specific case, the two cameras may besimultaneously used in the third mode, thereby acquiring a more accuratedepth image.

The function of the controller 606 may be performed by a control device(not shown) provided to the interface apparatus 12 separately from thecontent display apparatus 60.

FIG. 16 is a schematic diagram showing a second example of using thegesture-based user interface apparatus 12 according to the secondembodiment of the present invention. As shown in FIG. 16, if two or moreusers use the content display apparatus 60, the first depth camera 70and the second depth camera 80 are configured to recognize gestures ofdifferent users.

As shown, a first user 30 inputs a gesture using the first depth camera70 and a second user 90 inputs a gesture using the second depth camera80. In this case, the users 30 and 90 can acquire control rights for thedepth cameras 70 and 80 using specific operations. Each camerarecognizes only the user gesture having the control rights and outputs acontrol command corresponding thereto. Accordingly, it is possible toprevent the first depth camera 70 and the second depth camera 80 fromrespectively recognizing the gestures of the users without the controlrights.

FIG. 17 is a schematic diagram showing a third example of using thegesture-based user interface apparatus 12 according to the secondembodiment of the present invention. As shown in FIG. 17, a first depthcamera 72 is disposed to photograph an image at a front side of the userand a second depth camera 82 is disposed to photograph an image at aposition different from the front side of the user. The second depthcamera 82 may be a movable camera in order to provide various images inaddition to the front image of the user. In this case, the controller606 may control one or both of the first depth camera 72 and the seconddepth camera 82 to be used according to the above-described gesturerecognition mode.

FIG. 18 is a flowchart illustrating the gesture-based user interfacemethod according to the second embodiment of the present invention.

First, the gesture recognition mode of the user is determined (S200). Asdescribed above, the gesture recognition mode may be divided into afirst mode for recognizing a whole-body gesture and a second mode forrecognizing a gesture involving only a part of the body. The recognitionrange of the second mode is narrower than that of the first mode. Athird mode may be added according to the characteristics of content tobe reproduced.

Next, at least one of the first depth camera and the second depth camerais selected according to the gesture recognition mode (S210). Forexample, only the second depth camera which is the wide angle camera isused in the first mode and only the first depth camera which is the zoomcamera is used in the second mode. In a specific case, two cameras maybe simultaneously used in the third mode, thereby acquiring a moreaccurate depth image.

As shown in FIG. 16, if two or more users use the content displayapparatus 60, both the first depth camera 70 and the second depth camera80 may be used.

As shown in FIG. 17, if the first depth camera 72 is disposed tophotograph the image at the front side of the user and the second depthcamera 82 is disposed to photograph the image at the position differentfrom the front side of the user, one or both of the first depth camera72 and the second depth camera 82 may be used according to theabove-described gesture recognition mode.

Next, light is irradiated to the user, the light reflected from the useris received by the image sensor and the depth image is acquired (S220).The user gesture is recognized based on the depth image using the imageprocessor (S230), the control command corresponding to the recognizeduser gesture is output (S240), and the function corresponding to thecontrol command is performed with respect to the content displayapparatus (S250). A detailed description of the above-described steps isequal to that of the first embodiment and a description thereof will beomitted.

According to the present invention, it is possible to more accuratelyand efficiently recognize a user gesture according to conditions such asthe type of a program or content provided by a content display apparatusor the position of the user.

It is possible to minimize power consumption when recognizing a usergesture.

It is possible to minimize capacity of data transmitted and receivedwhen recognizing a user gesture so as to more rapidly recognize the usergesture.

It is possible to more accurately recognize a user gesture using two ormore camera devices.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the inventions. Thus, itis intended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

What is claimed is:
 1. An apparatus configured to recognize a gesture,comprising: a light source configured to irradiate light; an imagesensor configured to: receive a portion of the light that is reflectedfrom a user, convert the received light into a depth image of the user,and output the depth image; an image processor configured to recognizethe gesture of the user based on the depth image; a controllerconfigured to control the light source, the image sensor and the imageprocessor, wherein the controller is further configured to: decide agesture recognition mode based on at least one of a range of therecognized gesture and accuracy of the recognized gesture, wherein atleast one of an optical wavelength of the light source, an optical powerlevel of the light source and a resolution of the image sensor iscontrolled to change at least one of the range of the recognized gestureand the accuracy of the recognized gesture of the decided gesturedrecognition mode, wherein the gesture recognition mode includes a firstmode for recognizing a whole-body gesture and a second mode forrecognizing a gesture involving a part of the body, wherein arecognition range of the second mode is narrower than a recognitionrange of the first mode, wherein if the controller controls the framerate in the first mode, the frame rate is set to a first frame rate,wherein if the controller controls the frame rate in the second mode,the frame rate is set to a second frame rate, and wherein the secondframe rate is greater than the first frame rate.
 2. The apparatus ofclaim 1, wherein if the controller controls the optical wavelength inthe first mode, the optical wavelength is set to a first wavelength,wherein if the controller controls the optical wavelength in the secondmode, the optical wavelength is set to a second wavelength, and whereinthe second wavelength is shorter than the first wavelength.
 3. Anapparatus configured to recognize a gesture, comprising: a light sourceconfigured to irradiate light; an image sensor configured to: receive aportion of the light that is reflected from a user, convert the receivedlight into a depth image of the user, and output the depth image; animage processor configured to recognize the gesture of the user based onthe depth image; a controller configured to control the light source,the image sensor and the image processor, wherein the controller isfurther configured to: decide a gesture recognition mode based on atleast one of a range of the recognized gesture and accuracy of therecognized gesture, wherein at least one of an optical wavelength of thelight source, an optical power level of the light source and aresolution of the image sensor is controlled to change at least one ofthe range of the recognized gesture and the accuracy of the recognizedgesture of the decided gestured recognition mode, wherein the gesturerecognition mode includes a first mode for recognizing a whole-bodygesture and a second mode for recognizing a gesture involving a part ofthe body, wherein a recognition range of the second mode is narrowerthan a recognition range of the first mode, wherein if the controllercontrols the resolution of the depth image, the resolution of the depthimage is set to a first resolution, wherein if the controller controlsthe resolution of the depth image, the resolution of the depth image isset to a second resolution, and wherein the second resolution is higherthan the first resolution.
 4. The apparatus of claim 1, wherein thegesture recognition mode includes; a short distance mode in which adistance between the apparatus and the user is less than a referencedistance, and a long distance mode in which the distance is greater thanthe reference distance, wherein the controller is further configured toset the optical power level to a first power value in the short distancemode and set the optical power level to a second power value in the longdistance mode, and wherein the second power value is greater than thefirst power value.
 5. The apparatus of claim 1, further comprising: amemory configured to store information regarding a command correspondingto the recognized gesture of the user, wherein if the controllerrecognizes the gesture of the user, the controller is configured tooutput the command to a content display apparatus based on theinformation corresponding to the command.
 6. A method of recognizing agesture by an apparatus having a light source, an image sensor and animage processor, the method comprising: deciding a gesture recognitionmode based on at least one of a range of the recognized gesture andaccuracy of the recognized gesture, and controlling an operation of theapparatus according to the gesture recognition mode, the gesturerecognition mode including settings for controlling at least one of anoptical wavelength of the light source, an optical power level of thelight source, a frame rate of the light source and an image resolutionof the image sensor to change at least one of the range of therecognized gesture and the accuracy of the recognized gesture, whereinthe operation of the apparatus includes irradiating light by the lightsource; receiving a portion of the light that is reflected from a user;converting the received light into a depth image of the user by theimage sensor; outputting the depth image; and recognizing the gesture ofthe user based on the depth image, wherein the gesture recognition modeincludes a first mode for recognizing a whole-body gesture and a secondmode for recognizing a gesture involving only a part of the body,wherein a recognition range of the second mode is narrower than arecognition range of the first mode, wherein the step of controllingincludes setting the frame rate to a first frame rate in the first modeand setting the frame rate to a second frame rate in the second mode,and wherein the second frame rate is greater than the first frame rate.7. The method of claim 6, wherein the step of controlling includessetting the optical wavelength a first wavelength in the first mode andsetting the optical wavelength to a second wavelength in the secondmode, and wherein the second wavelength is shorter than the firstwavelength.
 8. A method of recognizing a gesture by an apparatus havinga light source, an image sensor and an image processor, the methodcomprising: deciding a gesture recognition mode based on at least one ofa range of the recognized gesture and accuracy of the recognizedgesture, and controlling an operation of the apparatus according to thegesture recognition mode, the gesture recognition mode includingsettings for controlling at least one of an optical wavelength of thelight source, an optical power level of the light source, a frame rateof the light source and an image resolution of the image sensor tochange at least one of the range of the recognized gesture and theaccuracy of the recognized gesture, wherein the operation of theapparatus includes irradiating light by the light source; receiving aportion of the light that is reflected from a user; converting thereceived light into a depth image of the user by the image sensor;outputting the depth image; and recognizing the gesture of the userbased on the depth image wherein the gesture recognition mode includes afirst mode for recognizing a whole-body gesture and a second mode forrecognizing a gesture involving only a part of the body, and wherein arecognition range of the second mode is narrower than a recognitionrange of the first mode, wherein the step of controlling includessetting the image resolution to a first resolution in the first mode andsetting the image resolution to a second resolution in the second mode,and wherein the second resolution is higher than the first resolution.9. The method of claim 6, wherein the gesture recognition mode includesa short distance mode in which a distance between the apparatus and theuser is less than a reference distance and a long distance mode in whichthe distance is greater than the reference distance, wherein the step ofcontrolling includes setting the optical power level to a first powervalue in the short distance mode and setting the optical power level isset to a second power value in the long distance mode, and wherein thesecond power level is greater than the first power value.
 10. The methodof claim 6, further comprising: storing information regarding a commandcorresponding to the recognized gesture of the user; and outputting thecommand corresponding to the recognized gesture of the user to a contentdisplay apparatus based on the information regarding the command.